Evaluating Probabilistic Matrix Factorization on Netflix Dataset

Collaborative Filtering attempts to make automatic taste recommendations by examing a large number of taste information. Methods for achieving Collaborative Filtering can be broadly categorized into model based, and memory based techniques. In this project, we review and implement three variants of Probabilistic Matrix Factorization, a model based Collaborative Filtering algorithm. We compare the performance of Probabilistic Matrix Factorization to a memory based Collaborative Filtering algorithm, the K nearest neighbor algorithm. The model performance is compared using three datasets derived from the full NetFlix movie dataset, each with varying data sparsity. Specifically, the root mean square error measure (RMSE) is used as the metric to compare the relative performance between the different CF algorithms. In our performance evaluation, we discovered when the data sparsity is sufficiently low, Probabilistic Matrix Factorization out performs K nearest neighbor, achieving RMSE of as low as 0.83. However, when the data sparsity is high, KNN yields marginally better performance, obtaining RMSE score of 1.0453 on the most sparse dataset, while the regular Probabilistic Matrix Factorization scored a 1.0771 on the same dataset